Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: attest integration #6963

Open
wants to merge 51 commits into
base: main
Choose a base branch
from

Conversation

hi-ogawa
Copy link
Contributor

@hi-ogawa hi-ogawa commented Nov 26, 2024

Description

Based on #6852, now I moved all code to core.

summary

Add new assertions:

  • toMatchTypeSnapshot/toMatchTypeInlineSnapshot
  • toMatchTypeErrorSnapshot/toMatchTypeErrorInlineSnapshot
  • toMatchTypeCompletionSnapshot/toMatchTypeCompletionInlineSnapshot

and a new option to "enable" them (without the option, assertions become no-op and no-error):

  • vitest --attest
  • or defineConfig({ test: { attest: true } })

Example on stackblitz https://stackblitz.com/edit/vitest-dev-vitest-ywts26?file=test%2Fbasic.test.ts

example code
test('inline', () => {
  expect(squared).toMatchTypeInlineSnapshot(`(n: number) => number`);

  expect(() =>
    // @ts-expect-error
    squared('foo')
  ).toMatchTypeErrorInlineSnapshot(
    `Argument of type 'string' is not assignable to parameter of type 'number'.`
  );

  expect(
    () =>
      // @ts-expect-error
      squared['a']
  ).toMatchTypeCompletionInlineSnapshot(`
    {
      "a": [
        "apply",
        "arguments",
      ],
    }
  `);
});

todo

tbd

  • config , default behavior, etc...
  • workspace support? (currently attest runs independently for each project with { attest: true })

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

Copy link

netlify bot commented Nov 26, 2024

Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit 2ee32da
🔍 Latest deploy log https://app.netlify.com/sites/vitest-dev/deploys/67459f0632e1c40008947a19
😎 Deploy Preview https://deploy-preview-6963--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

Copy link

pkg-pr-new bot commented Nov 26, 2024

@vitest/coverage-istanbul

npm i https://pkg.pr.new/@vitest/coverage-istanbul@6963

@vitest/expect

npm i https://pkg.pr.new/@vitest/expect@6963

@vitest/mocker

npm i https://pkg.pr.new/@vitest/mocker@6963

@vitest/browser

npm i https://pkg.pr.new/@vitest/browser@6963

@vitest/coverage-v8

npm i https://pkg.pr.new/@vitest/coverage-v8@6963

@vitest/pretty-format

npm i https://pkg.pr.new/@vitest/pretty-format@6963

@vitest/runner

npm i https://pkg.pr.new/@vitest/runner@6963

@vitest/snapshot

npm i https://pkg.pr.new/@vitest/snapshot@6963

@vitest/spy

npm i https://pkg.pr.new/@vitest/spy@6963

@vitest/ui

npm i https://pkg.pr.new/@vitest/ui@6963

@vitest/utils

npm i https://pkg.pr.new/@vitest/utils@6963

vite-node

npm i https://pkg.pr.new/vite-node@6963

vitest

npm i https://pkg.pr.new/vitest@6963

@vitest/web-worker

npm i https://pkg.pr.new/@vitest/web-worker@6963

@vitest/ws-client

npm i https://pkg.pr.new/@vitest/ws-client@6963

commit: 7d0bfd5

@hi-ogawa hi-ogawa changed the title feat: attest integration (wip) feat: attest integration Nov 26, 2024
@hi-ogawa hi-ogawa marked this pull request as ready for review November 27, 2024 03:50
@mrazauskas
Copy link

Perhaps it would be useful to add more complex examples? I tried to play with a recursive type:

import { expect, test } from 'vitest';

type JsonValue = string | number | boolean | JsonObject | Array<JsonValue>;

interface JsonObject {
  [key: string]: JsonValue;
}

function stringify(input: JsonValue): string {
  return JSON.stringify(input);
};

test('stringify', () => {
  expect(stringify).toMatchTypeInlineSnapshot(`(input: JsonValue) => string`);
});

Do you think this snapshot is good?

I mean, it will fail if name of the argument gets changed, although that’s not really important. But it will pass if type is redefined into type JsonValue = Record<string, unknown>, although this is exactly the change one would like to catch. Did I miss something?

JSX components were mention in #5857, so here is an attempt:

interface ButtonProps {
  text: string;
  type?: "reset" | "submit";
}
 
function Button({ text, type }: ButtonProps) {
  return <button type={type}>{text}</button>;
}

test('Button', () => {
  expect(Button).toMatchTypeInlineSnapshot(`({ text, type }: ButtonProps) => Element`);
});

Good to see names of props, but if type would be marked required, that will not get caught. Also note that JSX.Element got stringified as Element. Namespace is lost. I have noticed this previously in tests of DefinetelyTyped libraries. For instance, Immutable.Map and the built-in Map both get stringified as Map by dtslint.

@hi-ogawa
Copy link
Contributor Author

@mrazauskas Hi, first of all, thanks for testing out!

About what's printed in the snapshot, we've talked with arktype/attest author @ssalbdivad. If I remember correctly, that's what we can get from ts server api. Normally it should match with what you see when hovering on IDE, but vscode can somehow show namespace without getting stripped.

We're not familiar with this area, so mostly we'll need to delegate to what attest supports for now even if we integrate some features as Vitest builtin. I guess this feature will start as experimental and see if the representation makes sense to users in general.

@mrazauskas
Copy link

We're not familiar with this area

Fair enough. I mean.. Are you about to ship this feature knowing that you are not familiar with it?

@hi-ogawa
Copy link
Contributor Author

In terms of API, I'm mostly only using writeAssertionData from @ark/attest, which analyzes xxx inside expect(xxx) and then output .attest/assertions/typescript.json file.

This in turn uses ts.typeChecker.typeToString API from typescript. https://github.com/arktypeio/arktype/blob/649f70eb2ec48b5203099469c6b59d0b5fe29260/ark/attest/cache/ts.ts#L208-L230 If the limitation is typescript API level, then Vitest cannot change anything for the snapshot though 🤔

@hi-ogawa hi-ogawa added the p2-to-be-discussed Enhancement under consideration (priority) label Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cr-tracked p2-to-be-discussed Enhancement under consideration (priority)
Projects
Status: P2 - 2
Development

Successfully merging this pull request may close these issues.

Is it possible to add snapshots in type testing?
2 participants